Unsupervised Reinforcement Learning in Multiple Environments

Several recent works have been dedicated to unsupervised reinforcement learning in a single environment, which policy is first pre-trained with interactions, and then fine-tuned towards the optimal for several downstream supervised tasks defined over same environment. Along this line, we address problem of class multiple environments, interactions from whole class, any environment class. Notably, inherently multi-objective as can trade off pre-training objective between environments many ways. In work, foster an exploration strategy that sensitive most adverse cases within Hence, cast maximization mean critical percentile state visitation entropy induced by environments. Then, present gradient algorithm, alphaMEPOL, optimize introduced through mediated Finally, empirically demonstrate ability algorithm explore challenging classes continuous show greatly benefits w.r.t. scratch.